Role-Playing with Robot Characters: Increasing User Engagement through Narrative and Gameplay Agency
Data Analysis
Spencer Ng, Ting-Han Lin, You Li, Sarah Sebo
3.6.1 Short Flow State Scale: Challenge-Skill Balance
I felt like I was competent enough to meet the high demands of the situation.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.9 | 1.021 |
| Increased Gamification | 4.2 | 1.240 |
| Increased Narrative Agency | 4.4 | 0.883 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_1 by Q1.1
## Kruskal-Wallis chi-squared = 4.1922, df = 2, p-value = 0.1229## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_1 60 0.0385 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_1 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.35 -
## Increased Narrative Agency 0.18 1.00
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Action-Awareness Merging
I did things spontaneously and automatically without having to think.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.40 | 1.188 |
| Increased Gamification | 3.25 | 1.164 |
| Increased Narrative Agency | 3.45 | 1.146 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_2 by Q1.1
## Kruskal-Wallis chi-squared = 0.35427, df = 2, p-value = 0.8377## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_2 60 -0.0289 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_2 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Clear Goals
I had a strong sense of what I wanted to do.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.15 | 0.933 |
| Increased Gamification | 4.00 | 0.858 |
| Increased Narrative Agency | 4.10 | 1.071 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_3 by Q1.1
## Kruskal-Wallis chi-squared = 10.759, df = 2, p-value = 0.004611## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_3 60 0.154 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_3 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.021 -
## Increased Narrative Agency 0.013 1.000
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Unambiguous Feedback
I had a good idea while I was performing of how well I was doing.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.1 | 1.252 |
| Increased Gamification | 4.3 | 0.865 |
| Increased Narrative Agency | 3.7 | 1.081 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_4 by Q1.1
## Kruskal-Wallis chi-squared = 10.785, df = 2, p-value = 0.004551## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_4 60 0.154 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_4 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.0053 -
## Increased Narrative Agency 0.3787 0.1542
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Concentration on Task
I was completely focused on the task in hand.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.00 | 1.376 |
| Increased Gamification | 3.60 | 1.273 |
| Increased Narrative Agency | 3.95 | 1.050 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_5 by Q1.1
## Kruskal-Wallis chi-squared = 5.3067, df = 2, p-value = 0.07042## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_5 60 0.0580 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_5 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.475 -
## Increased Narrative Agency 0.069 1.000
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Sense of Control
I had a feeling of total control over what I was doing.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 2.10 | 1.165 |
| Increased Gamification | 2.85 | 1.182 |
| Increased Narrative Agency | 2.60 | 0.883 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_6 by Q1.1
## Kruskal-Wallis chi-squared = 5.87, df = 2, p-value = 0.05313## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_6 60 0.0679 eta2[H] moderate##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_6 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.10 -
## Increased Narrative Agency 0.17 1.00
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Transformation of Time
The way time passed seemed to be different from normal.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.20 | 1.196 |
| Increased Gamification | 3.60 | 0.821 |
| Increased Narrative Agency | 3.85 | 1.137 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_7 by Q1.1
## Kruskal-Wallis chi-squared = 4.1988, df = 2, p-value = 0.1225## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_7 60 0.0386 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_7 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.97 -
## Increased Narrative Agency 0.17 0.66
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Loss of self-consciousness
I was not concerned with how Agent Lee and Agent Jay perceived me.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.85 | 1.268 |
| Increased Gamification | 3.65 | 1.040 |
| Increased Narrative Agency | 3.45 | 1.468 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_8 by Q1.1
## Kruskal-Wallis chi-squared = 0.92159, df = 2, p-value = 0.6308## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_8 60 -0.0189 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_8 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Autotelic Experience
The experience was extremely rewarding.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.10 | 1.021 |
| Increased Gamification | 3.65 | 0.933 |
| Increased Narrative Agency | 4.05 | 0.605 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q5.1_9 by Q1.1
## Kruskal-Wallis chi-squared = 9.6989, df = 2, p-value = 0.007833## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q5.1_9 60 0.135 eta2[H] moderate##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q5.1_9 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.269 -
## Increased Narrative Agency 0.006 0.534
##
## P value adjustment method: bonferroni3.6.1 Short Flow State Scale: Overall Flow Score
Overall Flow Score
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.200 | 0.494 |
| Increased Gamification | 3.678 | 0.574 |
| Increased Narrative Agency | 3.728 | 0.469 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: flow_avg by Q1.1
## Kruskal-Wallis chi-squared = 11.247, df = 2, p-value = 0.003611## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 flow_avg 60 0.162 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$flow_avg and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.0378 -
## Increased Narrative Agency 0.0034 1.0000
##
## P value adjustment method: bonferroni3.6.2 Player Experience of Need Satisfaction: Autonomy
Autonomy
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.400 | 1.241 |
| Increased Gamification | 4.800 | 1.089 |
| Increased Narrative Agency | 4.983 | 0.908 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: PENS..Autonomy by Q1.1
## Kruskal-Wallis chi-squared = 18.06, df = 2, p-value = 0.0001198## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 PENS..Autonomy 60 0.282 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$PENS..Autonomy and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.00184 -
## Increased Narrative Agency 0.00043 1.00000
##
## P value adjustment method: bonferroni3.6.2 Player Experience of Need Satisfaction: Competence
Competence
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.20 | 1.045 |
| Increased Gamification | 5.60 | 1.358 |
| Increased Narrative Agency | 5.35 | 1.235 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: PENS..Competence by Q1.1
## Kruskal-Wallis chi-squared = 11.58, df = 2, p-value = 0.003059## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 PENS..Competence 60 0.168 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$PENS..Competence and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.0069 -
## Increased Narrative Agency 0.0214 1.0000
##
## P value adjustment method: bonferroni3.6.2 Player Experience of Need Satisfaction: Relatedness
Relatedness
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.133 | 1.235 |
| Increased Gamification | 4.633 | 1.556 |
| Increased Narrative Agency | 4.100 | 1.410 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: PENS..Relatedness by Q1.1
## Kruskal-Wallis chi-squared = 9.7705, df = 2, p-value = 0.007557## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 PENS..Relatedness 60 0.136 eta2[H] moderate##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$PENS..Relatedness and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.013 -
## Increased Narrative Agency 0.078 0.746
##
## P value adjustment method: bonferroni3.6.2 Player Experience of Need Satisfaction: Immersion
Immersion
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.617 | 1.091 |
| Increased Gamification | 4.694 | 0.952 |
| Increased Narrative Agency | 4.361 | 0.969 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: PENS..Presence.Immersion by Q1.1
## Kruskal-Wallis chi-squared = 8.9381, df = 2, p-value = 0.01146## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 PENS..Presence.Immersion 60 0.122 eta2[H] moderate##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$PENS..Presence.Immersion and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.012 -
## Increased Narrative Agency 0.174 0.800
##
## P value adjustment method: bonferroni3.6.3 Player Social Perceptions: Awkward
Awkward
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.6 | 1.667 |
| Increased Gamification | 3.7 | 1.559 |
| Increased Narrative Agency | 4.1 | 1.683 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q9.1_1 by Q1.1
## Kruskal-Wallis chi-squared = 3.2818, df = 2, p-value = 0.1938## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q9.1_1 60 0.0225 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q9.1_1 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.22 -
## Increased Narrative Agency 0.99 1.00
##
## P value adjustment method: bonferroni3.6.3 Player Social Perceptions: Vulnerable
Vulnerable
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 2.7 | 1.490 |
| Increased Gamification | 2.3 | 1.174 |
| Increased Narrative Agency | 2.9 | 1.714 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q9.1_3 by Q1.1
## Kruskal-Wallis chi-squared = 1.0508, df = 2, p-value = 0.5913## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q9.1_3 60 -0.0167 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q9.1_3 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.3 Player Social Perceptions: Watched
Watched
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.35 | 1.981 |
| Increased Gamification | 4.50 | 1.906 |
| Increased Narrative Agency | 5.10 | 1.944 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q9.1_2 by Q1.1
## Kruskal-Wallis chi-squared = 1.9225, df = 2, p-value = 0.3824## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q9.1_2 60 -0.00136 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q9.1_2 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1.00 -
## Increased Narrative Agency 0.53 0.94
##
## P value adjustment method: bonferroni3.6.4 Robotic Social Attributes Scale: Advisor Robot’s Warmth
Advisor Robot’s Warmth
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.492 | 1.025 |
| Increased Gamification | 4.325 | 1.146 |
| Increased Narrative Agency | 4.083 | 0.714 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Jay..RoSAS.Warmth by Q1.1
## Kruskal-Wallis chi-squared = 5.58, df = 2, p-value = 0.06142## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Jay..RoSAS.Warmth 60 0.0628 eta2[H] moderate##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Jay..RoSAS.Warmth and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.10 -
## Increased Narrative Agency 0.25 1.00
##
## P value adjustment method: bonferroni3.6.4 Robotic Social Attributes Scale: Peer Robot’s Warmth
Peer Robot’s Warmth
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.367 | 0.747 |
| Increased Gamification | 4.683 | 1.217 |
| Increased Narrative Agency | 4.775 | 0.788 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Lee..RoSAS.Warmth by Q1.1
## Kruskal-Wallis chi-squared = 2.1339, df = 2, p-value = 0.3441## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Lee..RoSAS.Warmth 60 0.00235 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Lee..RoSAS.Warmth and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1.00 -
## Increased Narrative Agency 0.31 1.00
##
## P value adjustment method: bonferroni3.6.5 Additional Robot and Experience Perceptions: Advisor robot is actively engaged
Agent Jay was actively engaged with me during the experience.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.30 | 1.625 |
| Increased Gamification | 5.05 | 1.701 |
| Increased Narrative Agency | 5.10 | 1.165 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q8.5_4 by Q1.1
## Kruskal-Wallis chi-squared = 3.32, df = 2, p-value = 0.1901## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q8.5_4 60 0.0232 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q8.5_4 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.41 -
## Increased Narrative Agency 0.31 1.00
##
## P value adjustment method: bonferroni3.6.5 Additional Robot and Experience Perceptions: Peer robot is actively engaged
Agent Lee was actively engaged with me during the experience.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.8 | 2.016 |
| Increased Gamification | 5.6 | 1.231 |
| Increased Narrative Agency | 5.4 | 1.188 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q8.3_4 by Q1.1
## Kruskal-Wallis chi-squared = 10.842, df = 2, p-value = 0.004424## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q8.3_4 60 0.155 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q8.3_4 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.0077 -
## Increased Narrative Agency 0.0355 1.0000
##
## P value adjustment method: bonferroni3.6.5 Additional Robot and Experience Perceptions: Given the chance, I would participate in this experience in my free time.
Given the chance, I would participate in this experience in my free time.
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.10 | 1.744 |
| Increased Gamification | 5.65 | 1.461 |
| Increased Narrative Agency | 5.60 | 1.273 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q6.1_2 by Q1.1
## Kruskal-Wallis chi-squared = 10.834, df = 2, p-value = 0.004441## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q6.1_2 60 0.155 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q6.1_2 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.015 -
## Increased Narrative Agency 0.015 1.000
##
## P value adjustment method: bonferroni3.6.6 Coded Free-Response: Peer Robot’s Impression is Positive
Peer Robot’s Impression is Positive
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.40 | 0.503 |
| Increased Gamification | 0.40 | 0.503 |
| Increased Narrative Agency | 0.45 | 0.510 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 0.13714, df = 2, p-value = 0.9337## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 1 1
## 2 Control vs. Increased Narrative Agency 1 1
## 3 Increased Gamification vs. Increased Narrative Agency 1 13.6.6 Coded Free-Response: Peer Robot’s Impression is Negative
Peer Robot’s Impression is Negative
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.15 | 0.366 |
| Increased Gamification | 0.15 | 0.366 |
| Increased Narrative Agency | 0.00 | 0.000 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 3.3333, df = 2, p-value = 0.1889## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 1.0000 1.0000
## 2 Control vs. Increased Narrative Agency 0.2308 0.6923
## 3 Increased Gamification vs. Increased Narrative Agency 0.2308 0.69233.6.6 Coded Free-Response: Peer Robot’s Impression is Neutral
Peer Robot’s Impression is Neutral
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.0 | 0.000 |
| Increased Gamification | 0.2 | 0.410 |
| Increased Narrative Agency | 0.1 | 0.308 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 4.4444, df = 2, p-value = 0.1084## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.1060 0.3181
## 2 Control vs. Increased Narrative Agency 0.4872 1.0000
## 3 Increased Gamification vs. Increased Narrative Agency 0.6614 1.00003.6.6 Coded Free-Response: Peer Robot’s Impression is a Mixture of Positive and Negative Attributes
Peer Robot’s Impression is a Mixture of Positive and Negative Attributes
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.45 | 0.510 |
| Increased Gamification | 0.25 | 0.444 |
| Increased Narrative Agency | 0.45 | 0.510 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 2.2562, df = 2, p-value = 0.3237## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.3203 0.9608
## 2 Control vs. Increased Narrative Agency 1.0000 1.0000
## 3 Increased Gamification vs. Increased Narrative Agency 0.3203 0.96083.6.6 Coded Free-Response: Advisor Robot’s Impression is Positive
Advisor Robot’s Impression is Positive
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.35 | 0.489 |
| Increased Gamification | 0.55 | 0.510 |
| Increased Narrative Agency | 0.80 | 0.410 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 8.2805, df = 2, p-value = 0.01592## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.3406 1.0000
## 2 Control vs. Increased Narrative Agency 0.0095 0.0286
## 3 Increased Gamification vs. Increased Narrative Agency 0.1760 0.52803.6.6 Coded Free-Response: Advisor’s Robot’s Impression is Negative
Advisor’s Robot’s Impression is Negative
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.25 | 0.444 |
| Increased Gamification | 0.00 | 0.000 |
| Increased Narrative Agency | 0.00 | 0.000 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 10.909, df = 2, p-value = 0.004277## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.0471 0.1414
## 2 Control vs. Increased Narrative Agency 0.0471 0.1414
## 3 Increased Gamification vs. Increased Narrative Agency 1.0000 1.00003.6.6 Coded Free-Response: Advisor’s Robot’s Impression is Neutral
Advisor’s Robot’s Impression is Neutral
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.20 | 0.410 |
| Increased Gamification | 0.45 | 0.510 |
| Increased Narrative Agency | 0.05 | 0.224 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 9.1304, df = 2, p-value = 0.01041## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.1760 0.5280
## 2 Control vs. Increased Narrative Agency 0.3416 1.0000
## 3 Increased Gamification vs. Increased Narrative Agency 0.0084 0.02513.6.6 Coded Free-Response: Advisor Robot’s Impression is a Mixture of Positive and Negative Attributes
Advisor Robot’s Impression is a Mixture of Positive and Negative Attributes
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.20 | 0.410 |
| Increased Gamification | 0.05 | 0.224 |
| Increased Narrative Agency | 0.15 | 0.366 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 2.0192, df = 2, p-value = 0.3644## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.3416 1
## 2 Control vs. Increased Narrative Agency 1.0000 1
## 3 Increased Gamification vs. Increased Narrative Agency 0.6050 13.6.6 Coded Free-Response: Feeling Positive About the Overall Experience
Feeling Positive About the Overall Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.60 | 0.503 |
| Increased Gamification | 0.95 | 0.224 |
| Increased Narrative Agency | 0.95 | 0.224 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 11.76, df = 2, p-value = 0.002795## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.0197 0.059
## 2 Control vs. Increased Narrative Agency 0.0197 0.059
## 3 Increased Gamification vs. Increased Narrative Agency 1.0000 1.0003.6.6 Coded Free-Response: Feeling Ignored About the Overall Experience
Feeling Ignored About the Overall Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.70 | 0.47 |
| Increased Gamification | 0.20 | 0.41 |
| Increased Narrative Agency | 0.45 | 0.51 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 10.101, df = 2, p-value = 0.006406## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.0036 0.0109
## 2 Control vs. Increased Narrative Agency 0.2003 0.6009
## 3 Increased Gamification vs. Increased Narrative Agency 0.1760 0.52803.6.6 Coded Free-Response: Feeling Not Immersed About the Overall Experience
Feeling Not Immersed About the Overall Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 0.45 | 0.510 |
| Increased Gamification | 0.10 | 0.308 |
| Increased Narrative Agency | 0.20 | 0.410 |
Chi Squared Test
##
## Pearson's Chi-squared test
##
## data: new.matrix
## X-squared = 6.9333, df = 2, p-value = 0.03122## Adjusted p-values used the bonferroni method.## comparison raw.p adj.p
## 1 Control vs. Increased Gamification 0.0310 0.0929
## 2 Control vs. Increased Narrative Agency 0.1760 0.5280
## 3 Increased Gamification vs. Increased Narrative Agency 0.6614 1.00003.6.7 Personality and Prior Experience: Extraversion
Extraversion
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.475 | 1.272 |
| Increased Gamification | 3.725 | 1.577 |
| Increased Narrative Agency | 3.725 | 1.409 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: tipi_extraversion by Q1.1
## Kruskal-Wallis chi-squared = 0.31015, df = 2, p-value = 0.8563## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 tipi_extraversion 60 -0.0296 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$tipi_extraversion and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.7 Personality and Prior Experience: Openness
Openness
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 5.125 | 1.234 |
| Increased Gamification | 5.325 | 1.139 |
| Increased Narrative Agency | 5.250 | 0.896 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: tipi_openness by Q1.1
## Kruskal-Wallis chi-squared = 0.52118, df = 2, p-value = 0.7706## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 tipi_openness 60 -0.0259 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$tipi_openness and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.7 Personality and Prior Experience: Role-Playing Experience
Role-Playing Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 3.90 | 2.174 |
| Increased Gamification | 4.15 | 1.872 |
| Increased Narrative Agency | 4.35 | 2.368 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q10.2_1 by Q1.1
## Kruskal-Wallis chi-squared = 0.73815, df = 2, p-value = 0.6914## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q10.2_1 60 -0.0221 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q10.2_1 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.7 Personality and Prior Experience: Cryptic Puzzle Experience
Cryptic Puzzle Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.30 | 1.658 |
| Increased Gamification | 4.50 | 1.469 |
| Increased Narrative Agency | 4.65 | 2.134 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q10.2_2 by Q1.1
## Kruskal-Wallis chi-squared = 0.73807, df = 2, p-value = 0.6914## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q10.2_2 60 -0.0221 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q10.2_2 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.7 Personality and Prior Experience: Robot Experience
Robot Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 2.65 | 1.531 |
| Increased Gamification | 3.00 | 1.864 |
| Increased Narrative Agency | 3.05 | 1.877 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q10.2_3 by Q1.1
## Kruskal-Wallis chi-squared = 0.46899, df = 2, p-value = 0.791## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q10.2_3 60 -0.0269 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q10.2_3 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 1 -
## Increased Narrative Agency 1 1
##
## P value adjustment method: bonferroni3.6.7 Personality and Prior Experience: Programming Experience
Programming Experience
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 4.95 | 2.114 |
| Increased Gamification | 4.00 | 2.616 |
| Increased Narrative Agency | 5.40 | 1.930 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: Q10.2_4 by Q1.1
## Kruskal-Wallis chi-squared = 3.0448, df = 2, p-value = 0.2182## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 Q10.2_4 60 0.0183 eta2[H] small##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$Q10.2_4 and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.98 -
## Increased Narrative Agency 1.00 0.28
##
## P value adjustment method: bonferroni3.6.8 Verbal Engagement: Prompted Utterances
Prompted Utterances
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 40.398 | 14.467 |
| Increased Gamification | 105.652 | 32.630 |
| Increased Narrative Agency | 100.849 | 24.847 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: total_prompted_utterance by Q1.1
## Kruskal-Wallis chi-squared = 37.229, df = 2, p-value = 8.24e-09## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 total_prompted_utterance 60 0.618 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum exact test
##
## data: df$total_prompted_utterance and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 5.2e-10 -
## Increased Narrative Agency 2.0e-09 1
##
## P value adjustment method: bonferroni3.6.8 Verbal Engagement: Unprompted Utterances
Unprompted Utterances
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 46.239 | 51.364 |
| Increased Gamification | 36.178 | 40.224 |
| Increased Narrative Agency | 78.001 | 49.231 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: total_unprompted_utterance by Q1.1
## Kruskal-Wallis chi-squared = 9.3018, df = 2, p-value = 0.009553## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 total_unprompted_utterance 60 0.128 eta2[H] moderate##
## Pairwise comparisons using Wilcoxon rank sum test with continuity correction
##
## data: df$total_unprompted_utterance and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.988 -
## Increased Narrative Agency 0.068 0.017
##
## P value adjustment method: bonferroni3.6.8 Verbal Engagement: Combined Utterances
Combined Utterances
Plot
Mean and Standard deviation:
| Mean | Standard Deviation | |
|---|---|---|
| Control | 86.637 | 57.997 |
| Increased Gamification | 141.830 | 64.597 |
| Increased Narrative Agency | 178.850 | 58.661 |
Kruskal-Wallis Test
##
## Kruskal-Wallis rank sum test
##
## data: total_time_utterance by Q1.1
## Kruskal-Wallis chi-squared = 22.026, df = 2, p-value = 1.649e-05## # A tibble: 1 × 5
## .y. n effsize method magnitude
## * <chr> <int> <dbl> <chr> <ord>
## 1 total_time_utterance 60 0.351 eta2[H] large##
## Pairwise comparisons using Wilcoxon rank sum exact test
##
## data: df$total_time_utterance and df$Q1.1
##
## Control Increased Gamification
## Increased Gamification 0.002 -
## Increased Narrative Agency 1.7e-05 0.098
##
## P value adjustment method: bonferroni